DIF in Personality Assessment: Does Cognitive Ability Influence Item Interpretation?

PreVisor The current study proposes and illustrates a method for examining item complexity during test construction using item response theory. We examined whether differential item functioning existed within 4 personality scales across cognitive ability groups. Results indicated that personality items do differentially function across highly disparate cognitive ability groups. Organizations are increasingly relying on personality measures to make personnel decisions (Rothstein & Goffin, 2006). Various meta-analyses have established a link between conscientiousness pervasive finding in selection research is that cognitive ability (CA) test scores have trivially small correlations with measures of personality (e. This is a desirable trait of a selection procedure as uncorrelated predictors tend to account for more incremental variance in the criteria than do correlated predictors (Schmidt & Hunter, 1998). Despite the lack of manifest correlation between personality and CA, a lack of correlation does not mean that CA cannot impact the measurement of personality or other survey measures. In this study, we examined the invariance of a series of personality measures for respondents with different levels of CA. Our purposes for doing so were twofold: (a) to examine if there is an impact of CA on personality measurement, and (b) to illustrate a technique that can be used to identify items that are overly complex. To accomplish these goals, we employ item response theory (IRT) methods of detecting differential functioning (DF) across low and high cognitive ability groups. The Response Process In order to understand how CA may impact personality and other survey-format measures, it is important to fully understand the survey response process. While there are several models of response, there is a consensus that the process involves at least the following stages: literally understanding the item, understanding the pragmatic interpretation of the item, retrieving relevant information from memory, forming a judgment, and mapping the judgment to the response options. In the first stage, the respondent must interpret the semantic meaning of an item, which requires " the identification of words, the recall of lexical information from semantic memory, and the construction of a meaning of the utterance " (Schwarz, 1999, p 94). The respondent must also interpret the pragmatic meaning of a question, or what it is that the survey is asking. This requires the respondent to make assumptions about the intended meaning of the survey item which may go beyond the literal description (Schwarz, 1999). The third stage involves a search of short …